Multivariate-state hidden Markov models for simultaneous transcription of phones and formants

A multivariate-state HMM | an HMM with a vector state variable | can be used to nd jointly optimal phonetic and formant transcriptions of an utterance. The complexity of searching a multivariate state space using the BaumWelch algorithm is substantial, but may be signi cantly reduced if the formant frequencies are assumed to be conditionally independent given knowledge of the phone. Operating with a known phonetic transcription, the multivariatestate model can provide a maximum a posteriori formant trajectory, complete with con dence limits on each of the formant frequency measurements. The model can also be used as a phonetic classi er by adding the probabilities of all possible formant trajectories. A test system is described which requires only nine trainable parameters per formant per phonetic state: ve parameters to model formant transitions, and four to model spectral observations. Further simpli cations were achieved through parameter tying.